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Introduction 


The COVID-19 pandemic has given the entire country a crash course in virtual education and 
digital education platforms. As school buildings closed in the spring, education technology 
vendors immediately offered educators free products.' Additionally, such vendor corpora- 
tions, as well as tech industry trade associations, venture capitalists and venture philanthro- 
pists who have been promoting virtual education for over a decade, all worked quickly to 
position digital programs and platforms as the obvious solution for schools that had to close 
buildings to avoid transmitting the virus.” 3 These actors are promoting digital options not 
only as schools’ go-to response to the crisis, but also as a leap forward into the new normal 
for the core education infrastructure in a radically altered school environment when the 
crisis is over.* 


Unfortunately, state policymakers, communities, and district and school administrators 
have little information other than marketing materials to use in evaluating the claims tech- 
nology vendors and other promoters make about virtual learning. Such claims can be extrav- 
agant and promotional materials seductive, but reality often contradicts them. 


Meanwhile, teachers, students, and parents have struggled with mixed success to adjust to 
virtual-education technologies. Many students and parents have been sidelined altogether 
because they lack access to broadband, computers, and other digital necessities. Parents 
also often lack the time, resources, and knowledge required to meaningfully engage in the 
technological programming offered. In addition, students’ privacy is undermined by federal 
laws that allow technology companies to be legally defined as school officials and by state 
laws that exempt personalized or adaptive learning products from privacy protections.5 


With the pandemic creating a surge in demand for virtual education, decision-makers face 
an urgent need to get digital platforms and programs up and running in schools. What re- 
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search evidence there is, however, does not support claims that virtual education produces 
desired student outcomes, as compared to conventional face-to-face approaches to teaching 
and learning. Online schools, in particular, have yielded very poor outcomes.® Moreover, the 
use of digital platforms and learning programs is tied to significant threats to the integrity 
of schools’ curriculum and instruction programs, their student assessments, and their data 
collection and record-keeping practices.’? Compared to the surface transparency of tradi- 
tional textbooks, tests, and record books, there is a lot “hidden under the hood” of virtual 
technologies. 


Purpose of This Collection 


In this pandemic, school leaders are forced to consider a set of very imperfect options as 
they struggle to reopen their schools. This three-brief collection identifies key issues for 
school leaders to consider before adopting a digital platform or learning program that will 
impact curriculum and teaching, student assessment, and privacy/data security. We do not 
review specific programs, nor do we provide advice about which programs to adopt. 


Each brief in this collection can be used alone or in conjunction with one or both of its com- 
panion briefs. To allow for such flexible use, each includes recommendations unique to its 
specific focus as well as recommendations common across the set. 


The framing principle underlying all three briefs is that school leaders should ensure that 
any digital technology adopted reflects, rather than undermines or distorts, the school’s 
stated values and goals. In the context of the COVID-19 pandemic, the best many school 
leaders can do is minimize any potential harm that may result from the need to hastily adopt 
digital technologies. With this in mind, we offer the following additional principles to guide 
decision-making. 


Digital learning programs and platforms are less likely to harm students to the extent that 
they: 


¢ Retain curriculum and teaching practices consistent with school goals and values; 
Have been reviewed for bias by independent experts; 


Maintain teachers’ control of educational decisions rather than transfer those deci- 
sions to algorithms programmed into applications; 


Collect a minimal amount of student data; and 
¢ Prevent the transfer of student data to vendors and other unknown parties. 


These principles, in conjunction with the considerations detailed in each brief, can be used 
to help determine which products to choose, how to best use them in the current crisis, and 
which to abandon when the crisis has passed. 


http://nepc.colorado.edu/publication/virtual-learning 


Notes and References Introduction 


Schaffhauser, D. (2020, June ). Updated: Free resources for schools during COVID-19 outbreak. THE Journal. 
Retrieved August 18, 2020, from https://thejournal.com/articles/2020/03/13/free-resources-ed-tech- 
companies-step-up-during-coronavirus-outbreak.aspx 


Boninger, F., Molnar, A., & Saldana, C.M. (2019). Personalized learning and the digital privatization of 
curriculum and teaching (pp. 45-48). Boulder, CO: National Education Policy Center. Retrieved July 13, 2020, 
from http://nepc.colorado.edu/publication/personalized-learning 


Boninger, F., Molnar, A., & Saldafia, C. (2020). Big claims, little evidence, lots of money: The reality behind 
the Summit Learning Program and the push to adopt digital personalized learning platforms. Boulder, 
CO: National Education Policy Center. Retrieved July 13, 2020, from http://nepc.colorado.edu/publication/ 
summit-2020 


Saltman, K.J. (2018). The swindle of innovative educational finance. Minneapolis, MN: University of 
Minnesota Press. 


Williamson, B., & Hogan, A. (2020, July). Commercialisation and privatisation in/of education in the context 
of Covid-19. Brussels, Belgium: Education International. Retrieved August 18, 2020, from https://issuu.com/ 
educationinternational/docs/2020_eiresearch_gr_commercialisation_privatisation?fr=sZDJkYjJE1L1ODA2MTQ 


HolonIQ, a company that researches the international education technology market, reported over $32 billion 
in venture capital investment in educational technology between 2010-2019, and predicts over $87 billion to 
be invested between 2020-2030. 


HolonIQ (2020). EdTech started the decade with $500m of Venture Capital investments in 2010 and finished 
14x higher at $7B in 2019. We expect over $87bn to be invested over the next 10 years, almost triple the prior 
decade. Retrieved August 20, 2020, from https: //www.holoniq.com/notes/87bn-of-global-edtech-funding- 
predicted-to-2030/ 


From June 2016 through December 2018, Audrey Watters posted a series of blogposts on the education 
technology industry and its connection to venture capital. Find those posts here: 


Watters, A. (2018, December 26). The education technology industry network. Hack Education [blog]. 
Retrieved July 13, 2020, from http://network.hackeducation.com/blog/_ 


Williamson, B., & Hogan, A. (2020, July). Commercialisation and privatisation in/of education in the context 
of Covid-19. Brussels, Belgium: Education International. Retrieved August 18, 2020, from https://issuu.com/ 
educationinternational/docs/2020_eiresearch_gr_commercialisation_privatisation?fr=sZDJkYJE1LODA2MTQ 


Boninger, F. and Molnar, A. (2016). Learning to be watched: Surveillance culture at school—The eighteenth 
annual report on schoolhouse commercializing trends, 2014-2015 (pp. 14-16). Boulder, CO: National Educa- 
tion Policy Center. Retrieved August 20, 2020, from https://nepc.colorado.edu/publication/schoolhouse-com- 
mercialism-2015 


The National Education Policy Center has produced research reports on the performance of virtual schools 
since 2013. They are all available at https://nepc.colorado.edu/publications/research-briefs. Find the 2019 
NEPC report on virtual schools at: 


Molnar, A., Miron, G., Elgeberi, N., Barbour, M.K., Huerta, L., Shafer, S.R., & Rice, J.K. (2019). Virtual schools 
in the U.S. 2019. Boulder, CO: National Education Policy Center. Retrieved July 9, 2020, from http://nepc. 
colorado.edu/publication/virtual-schools-annual-2019 


Boninger, F., Molnar, A., & Saldana, C.M. (2019). Personalized learning and the digital privatization of 


http://nepc.colorado.edu/publication/virtual-learning 


curriculum and teaching (pp. 45-48). Boulder, CO: National Education Policy Center. Retrieved July 13, 2020, 
from http://nepc.colorado.edu/publication/personalized-learning 


Boninger, F., Molnar, A., & Saldafia, C. (2020). Big claims, little evidence, lots of money: The reality behind 
the Summit Learning Program and the push to adopt digital personalized learning platforms. Boulder, 
CO: National Education Policy Center. Retrieved July 13, 2020, from http://nepc.colorado.edu/publication/ 
summit-2020 


http://nepc.colorado.edu/publication/virtual-learning 


NEPC 


NATIONAL EDUCATION 
POLICY CENTER 


SECTION I 


CURRICULUM AND TEACHING ISSUES TO CONSIDER BEFORE 
ADOPTING A DIGITAL PLATFORM OR LEARNING PROGRAM 


Faith Boninger and Alex Molnar 
University of Colorado Boulder 


September 2020 


Even before the COVID-19 pandemic, the effort to provide students with meaningful cur- 
riculum and empower teachers to make the best use of their professional skills was under 
threat, as the result of two decades of test-heavy U.S. school reforms. The intense testing 
regime ushered in by No Child Left Behind rewarded students, teachers, and administrators 
when student memorization of facts translated, in the short term, into high test perfor- 
mance.' Against this backdrop, over the last decade, foundations and corporate interests 
have pushed aggressively to spread virtual technologies in schools.” Most recently, the push 
for virtual education has been coupled with a tech-friendly digitalized version of “personal- 
ized learning.” California-based Summit Schools, for example, armed with almost $200 mil- 
lion from the Gates Foundation, the Chan Zuckerberg Initiative, and others, had by 2018- 
2019 signed up almost 400 “partner schools” to use its “Summit Learning Program.” 


Without careful decision-making by school leaders, digital platforms and learning programs, 
with their focus on continuous assessment and testing, can undermine teachers’ ability to 
organize curricula responsive to student needs and to adopt instructional approaches that 
encourage higher-level thinking among their students.4 This problem is likely to worsen 
as larger numbers and more diverse groups of students are funneled into virtual educa- 
tion. The COVID-19 pandemic has dramatically increased the pressure on schools to quickly 
adopt virtual technologies and digital platforms despite a general lack of research evidence 
to guide their adoption.5 


To help school leaders make thoughtful decisions about digital platforms and learning pro- 
grams in general, and particularly in the current high-pressure environment, we discuss 
seven key issues to consider. 
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Pedagogical theories embedded in digital platforms and learning pro- 
grams shape the student learning environment. 


Many digital platforms implement some form of competency-based education (CBE, also 
known as competency-based learning or mastery-based learning). This approach has roots 
in the behaviorist psychology and “programmed instruction” popular in the 1950s and 
1960s. These popularized the idea that knowledge could be chopped up and delivered, like 
a product or commodity, via “teaching machines.” Advocates argued not only that students 
could “acquire” these bits of knowledge, but also that their ability to provide the required 
response to questions about each bit demonstrated their competency/mastery of it—and 
therefore their “learning.”® Although analog teaching machines did not take off as their in- 
ventors hoped, the marketing of digital “teaching machines” has been better funded, more 
persistent, and more successful.” 


Understanding learning as the acquisition of discrete bits of information and discrete skills 
limits how teachers, students, and administrators interact by defining what “counts” and 
what is important.® It encourages everyone in the school community to think and talk about 
students’ schoolwork—including their social-emotional development—in the context of their 
individual mastery of specific skills that will be useful to them.® These days, that is almost 
always narrowed to skills that are perceived to be in some way test performance- and job-re- 
lated. When teaching these skills is pre-loaded into a digital platform, it scripts the teaching 
and learning process. It crowds out the kind of unanticipated teaching moments that cannot 
be coded into any software, on which teachers can capitalize even when they are not in their 
lesson plan. 


Schools can create environments—cultures of learning and thinking—that encourage mean- 
ingful learning as an integral part of daily life..° Researchers and program designers in- 
creasingly recognize that programs to teach thinking cannot just be “implemented,” but 
rather must be established and cultivated within a social context." This means that effective 
teaching is not limited to specific classroom lessons, but also takes place spontaneously in 
the classroom and school as teachers both create school and classroom environments that 
support student learning and also capitalize on situations that arise outside of planned les- 
sons. This kind of teaching and learning may be undermined by digital products that shape 
the learning environment and structure learning opportunities to meet the requirements of 
that digital environment. 


The more that teaching and learning are shaped by the collection and use of easily quanti- 
fiable data points, the more narrow and limited the curriculum and definitions of “achieve- 
ment” will become, because boundaries of what is valued will be defined by those things that 
can best be captured and sorted electronically.” 


“Personalized learning” in digital platforms and learning programs 
does not necessarily result in personalized learning. 


“Personalized learning” has been aggressively promoted by the Gates Foundation and others 
for over a decade.'? There is no common definition of what “personalized learning” means,“ 
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although advocates for the approach tend to point to broad goals and assert that their peda- 
gogical approaches will meet the needs, strengths, and interests of each learner.'5 Although 
not all personalized learning is digital, the idea of personalizing learning has been the domi- 
nant rationale supporting the use of digital platforms and learning programs. Such products 
allow for students to advance through materials at their individual pace—with the ability to 
move forward through lessons dependent on assessment data. 


In contrast, common sense suggests that the term “personalized learning” implies a humane 
school and classroom environment and open, flexible teaching strategies. But this is a far 
cry from contemporary personalized learning programs, including the digital platforms de- 
signed to implement them, which often share the assumptions of competency-based educa- 
tion (CBE). That is, they conceptualize learning as a hyper-rational process of remembering 
facts and demonstrating specified skills according to a logically defined plan. Thus, digital 
“personalized” learning programs can limit students’ learning by channeling it into the kind 
of narrow, logical pathways that can be easily assessed by digital platforms.'® 


The mastery-based approach to learning and the capability built into some digital plat- 
forms for students to set and achieve individual learning goals may appear on the surface 
to be child-centered. However, the choices students are allowed to make are not necessarily 
meaningful. In many cases, the truly meaningful choices are made by software designers 
and developers who determine the content that students must master and how they must 
demonstrate that they have mastered it.” Algorithms determine how assessments are scored 
and how students will be nudged in particular directions. 


The Summit Learning Program, for example, embodies this type of hyper-rational, mas- 
tery-based approach not only to students’ learning of facts (i.e. “content knowledge”), but 
also, explicitly, to their academic and social and emotional development (i.e., “cognitive 
skills” and “habits of success”). According to Diane Taverner, CEO of Summit Public Schools, 
which created the digital Summit Learning Program: 


..1f you think about going into the platform, this is...where you are going to in- 
terface with your courses and your grades and all of the learning materials and 
where you'll take and submit your work and your assessments and so it’s a full 
comprehensive ... space where that happens and takes place.*® 


In other words, “personalization” in a program such as the Summit Learning Program is de- 
fined by its digital platform. Students in schools that adopt the program use the platform to 
choose their curricular materials (often from third-party websites), do their work, and take 
their tests. They also set goals and interact with their teachers on the platform. In short, 
the platform still manages all aspects of the circumbscribed student experience and tracks 
the “measurable outcomes.” 


In this way digital programs such as the Summit Learning Program force students—regard- 
less of their learning style—to engage with every aspect of their school life via the platform. It 
is not surprising that some students have expressed experiencing anxiety when their schools 
adopted the Summit Learning Program.”° When a program or platform promises “person- 
alized” learning for students, then, it woud be wise for school leaders to take a close look 
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at whether the term translates to any meaningful learning options for students with widely 
varying needs, interests, habits, and challenges. 


Algorithms embedded in digital platforms and learning programs shape 
teaching and curriculum. 


Algorithms represent theories about which pieces of information their authors consider 
valuable and how their authors believe those pieces of information should be assembled to 
draw conclusions. Therefore, it is essential to understand how algorithms in a particular 
product reflect inferences drawn about students and their learning. 


Algorithms are central to the day-to-day functioning of digital platforms and educational 
programs. They implement the regular formative assessments designed to mediate between 
teachers and children, and to influence children’s experience of the curriculum. In some pro- 
grams, the assessment is straightforward and teachers decide what and how students learn. 
In other programs, the assessment is less transparent: Teachers may not see the questions 
that their students are asked to answer while they work within the program, or understand 
why students received the grades they did. Yet those programs require teachers to, “in real 
time,” adjust their teaching to the assessment results that the algorithms report. Programs 
that feature “adaptive” or “personalized” learning bypass teachers completely and automate 
the instructional decision-making that teachers would ordinarily control. 


The more that a digital platform or learning program inserts itself into the relationship be- 
tween students and teachers, the more opportunities there are for its output to be flawed, 
and the greater the influence of those flaws is likely to be on how students are taught and 
assessed. The less that it is programmed to do, the less problematic it has opportunity to be. 


Cultural and other biases may be embedded in digital platforms and 
learning programs. 


Like any textbook or other physical curriculum or assessment material, algorithms may re- 
flect values or assumptions that may be second nature to the social demographic of their 
writer but not to members of other demographic groups. Biased descriptions, examples, or 
test questions are easier to identify on a written page, however, than in a digital platform or 
learning program where they disappear quickly from the screen. Biased decision-making by 
an algorithm embedded in a learning program (for example, one that marks as “incorrect” 
answers written in dialects other than standard American English) are completely hidden. 
Teachers, students, parents and community members are, therefore, less able to identify 
problems with them.” 


Machine-learning algorithms, in particular, reflect any bias in the data used to “train” them. 
For this reason, they have been found to have different accuracy rates for different demo- 
graphic groups, and to make different decisions when applied to different populations.?* A 
2016 ProPublica investigation, for example, found that algorithms purported to predict pris- 
on inmates’ likelihood of recidivism were more likely to be inaccurate when they assessed 
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Black as compared to White inmates.*3 In another example, Safiya Noble found that even 
seemingly objective Google search algorithms perpetuate harmful stereotypes about women 
and minorities.*4 Other authors have explored the dangers of relying on opaque algorithms 
to make consequential decisions about people’s lives in such domains as employment, career 
advancement, health, credit, and education.”5 


Although independent algorithmic audits can identify algorithmic bias, technology compa- 
nies are disincentivized from doing them because such audits may reveal the need for costly 
and time-consuming revision of their programs, and might cost them customers.*° Without 
independent audits of the opaque algorithms that run digital platforms and learning pro- 
grams, school leaders are forced to accept on faith that the conclusions those algorithms 
generate are valid. For these reasons, school leaders should ask questions about the algo- 
rithms that run the digital platforms and learning programs they are considering. Programs 
that have gone through an algorithmic audit are preferable to those that have not. 


Digital platforms and learning programs may socialize children to ac- 
cept surveillance. 


It becomes “common sense” to children who have been raised under constant surveillance 
that such surveillance is normal and natural, and that it is a fair price for getting services 
they want—especially because they cannot avoid it even if they wanted to. Two corollary ten- 
dencies accompany the assumption of ubiquitous surveillance. One is to trust the providers 
of digital services and not balk at giving away their private information to people or entities 
they do not know for uses they cannot identify. The other is to conform—to become self-con- 
scious in the presence of recording devices and suppress, rather than give voice to and de- 
velop—ideas or viewpoints that they suspect may not be normative.”’ Social psychological 
research suggests that surveillance makes people less open to new ideas, more anxious, less 
creative, and generally more conservative in their thinking.?® Much of that research was 
conducted on young adults in relatively transient settings, not on developing children over 
long periods of time. The prospects of how the effects might multiply in latter settings are 
very concerning. 


All children, including teens, are more susceptible than adults to having their affinities 
shaped by marketers exploiting their vulnerabilities. Because they believe that what their 
schools do and parents allow is in their best interest, children are growing up experiencing 
constant surveillance as a norm to be accepted and even welcomed into their academic and 
social lives, as it brings them both what they need and what they want from the Internet. 


Digital platforms and learning programs may expose students to mar- 
keting and behavioral tracking. 


While it is true that a lot of online advertising to children takes place outside the school set- 
ting, schools serve as a portal to and reinforcer of digital marketing media and messages.”? 
Let’s follow an imaginary high school student, D.J., to see how this might happen. 
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D.J.’s schoolwork puts her online for much of her day, where she seamlessly transitions be- 
tween school-assigned and commercial websites. How might this affect her? It starts with 
D.J. preparing an assignment for a class, let’s say a presentation about a book she read for 
her English class. As she moves in and out of the protected applications that are part of Goo- 
gle’s G Suite for Education, marketing companies quietly but persistently track her activity. 


YouTube is not one of the “core” products in Google’s suite of education applications. How- 
ever, this matters very little since it is one of the most popular third-party sites to which 
students are sent by educational products.3° Accompanied by an application that identifies 
tracking, we surfed through other sites students might be likely to visit. We found 16 compa- 
nies tracking us from dictionary.com and over 35 from Sparknotes.*! With the information 
they collect about her, these companies—or other companies to whom they sell her data— 
determine what kinds of ads D.J. might respond to, and serve them to her on those sites and 
on others she visits. 


By feeding children ads and other content personalized to appeal specifically to them, and 
also by choosing what not to show them, marketers influence children’s thoughts, feelings, 
and behaviors.*? As they do, they also test, adjust, and perfect their models of influence—and 
then track and target some more.*3 They do it repeatedly from the time D.J. or any student 
starts using the Internet. Unless schools are vigilant, schoolwork will help marketers hold 
children in an environment in which their interests, attitudes, and anxieties are shaped 
carefully over time by repeated exposure to commercial messages in a virtual environment 
that surrounds them with products and ideas not designed to promote their healthy devel- 
opment, but rather to push them to purchase something.*4 


Digital platforms and learning programs offered by public sources may 
be preferable to those offered by private vendors. 


It is a given that a for-profit corporation will focus on its bottom line—and that the programs 
it provides to schools must benefit that bottom line. The tension between the educative mis- 
sion of schools and the corporate imperative to earn profits means that when corporations 
enter the schools, there is going to be pressure to create student experiences and shape 
student attitudes in ways that support, or at least do not undermine, corporate profitability. 


An important goal of corporations that promote digital educational products is to create a 
consumer base for their commercial products. Another is to generate data that can be sold 
to advertisers and others. As software tracks children, it creates opportunities for companies 
to develop profiles on them that may be used for targeted marketing while at the same time 
accustoming students to take being tracked for granted.°5 


Districts that have developed their own digital learning approaches are not motivated to 
create a consumer base for their products or to generate data from which they may profit. 
They are therefore less likely to integrate consumer-culture values into their platforms and 
educational programs or to promote consumption, and more likely to limit and to better 
safeguard the data they collect. 
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Research Landscape Related to Digital Platforms and/or Learning 
Platforms in a Virtual Environment: Curriculum and Teaching 


Michael K. Barbour, Touro University California 


Research on virtual education is very limited.2° What little exists often focuses on comparisons be- 
tween the virtual school environment and face-to-face settings. One such comparison, for example, 
consistently found that most virtual schools had a student-teacher ratio that was two to three times 
the national average for brick-and-mortar schools.*” Because of the high student-teacher ratios in 
virtual schools, they tend to rely on algorithms built into digital platforms to organize content, struc- 
ture pedagogy, and administer and evaluate student assessments. Further, they generally rely upon 
parents/guardians not only to supervise, but also to play a significant role in the preparation and 
delivery of instruction.%® 


The parents’ role begins with the time they must spend preparing and planning the next day’s in- 
structional material for their students, and it continues throughout the day.39 A 2006 Wisconsin 
Appeals Court decision noted the various activities required of parents forces them to devote four to 
five hours per day to educating their child.4° Gerald Bracey neatly summarized the situation when he 
wrote that although the students are enrolled in a virtual school, most children are homeschooled.* 
This situation creates an increased reliance on both the digital platform’s algorithms and the online 
content that it delivers. 


Very little is known about the daily life of students attending virtual schools, because the informa- 
tion available is usually both dated and provided by either the corporate educational management 
organizations themselves or secondhand reviews.4? There has been no public external review of the 
nature of virtual schools’ curriculum in over a decade. In 2001, Trotter described the online curric- 
ulum as “typical worksheet-style computer lessons, with brief bits of animation or sound effects as 
rewards.”43 In 2004, Bracey concluded that “the curriculum is not interesting and it promotes a one- 
size-fits-all approach. The instruction is mechanical and the system does not encourage creativity.”44 
In 2005, Baker and his colleagues indicated that the online curriculum “emphasizes phonics-based 
reading and a great book approach in literature [and an early foundation in basic arithmetic]. In 
social studies, Western culture and history is emphasized.”*5 Such descriptions led Ohanian to con- 
clude in 2004 that the online curriculum of many virtual schools contradicts the commonly accepted 
understanding “that children learn more effectively in environments that allow them to work inde- 
pendently and with each other to construct their own knowledge.”4° It would be expected that over 
the past 15 years the online curriculum has improved, but anecdotal evidence suggests otherwise.*” 
However, it is important to note that beyond these cited works conducted when full-time virtual 
schools first began operating, there continues to be an absence of independent research into the in- 
structional exchange and the online curriculum of these virtual schools.4* Some have speculated this 
absence is due to the for-profit nature of the corporations that operate the virtual schools serving the 
majority of students.49 


Some research has suggested that students engaged in supplemental virtual schooling have better 
outcomes than students engaged in full-time virtual schools.5° There are a few intertwined explana- 
tions for this difference. The student-teacher ratio in most supplemental virtual courses is similar to 
that of brick-and-mortar classrooms. While this hardly ensures that teachers will rely less on the 
online curriculum and the restrictions imposed by a digital platform, it creates the possibility that 
they might. Also, because the online curriculum of supplemental virtual education is more frequently 
designed by a teacher or team of teachers,** it tends to have less of a behaviorist approach than the 
curriculum of full-time virtual schools. 


http://nepce.colorado.edu/publication/virtual-learning 


Conclusion 


Adopting commercial digital platforms and learning programs can pose real risks to the 
integrity of schools’ curriculum and teaching. School and district leaders can minimize the 
risks by judiciously choosing and using products they adopt. To minimize risks, it is import- 
ant that the values and goals of school educational programs frame the decision-making 
process. Digital platforms and learning programs should not drive the curriculum, pedago- 
gy, assessment, or data collection and record-keeping practices of the schools. We recom- 
mend that school and district leaders: 


¢ Define the pedagogical values, goals, and practices they hope to achieve before 
considering the adoption of a particular digital educational product; 


Clarify the ways in which any digital educational product would advance their 
self-defined values, goals, and practices; 


Identify potential negative consequences—in this case, for curriculum and teach- 
ing—that may be associated with the use of that product and devise strategies for 
avoiding them; 


Determine which of their defined values, goals, and practices can be best achieved 
by non-digital means and which require digital means; 


As they assess the suitability of any particular product, we recommend that they consider: 
¢ How the product shapes the learning environment for students; 


What terms such as “personalized learning” mean in practice in the context of 
the product; 


The impact of algorithms embedded in the product on teaching and curriculum; 


Cultural and other biases that may be embedded in the algorithms; 
Whether and how the product teaches critical thinking; 
How the product may socialize children to accept surveillance; 


How the product may expose students to marketing and behavioral tracking; 
and 


If the product was produced by a public source or a private vendor. 
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The passage of No Child Left Behind in 2002 helped frame the virtual education choices 
schools now have. Promoted by the lobbying efforts of tech-friendly foundations such as 
the Bill and Melinda Gates Foundation and corporations such as Pearson, the emphasis on 
standardized tests and continuous student assessment contributed significantly to the de- 
mand for “ed tech” in schools. It takes computers to process the massive amount of test data 
schools are required to collect and report. 


The tech industry and a host of self-interested vendors and corporations have further stoked 
the demand for computers by aggressively promoting virtual education over the last decade 
and a half.t The Gates Foundation and Chan Zuckerberg Initiative, in particular, have spent 
hundreds of millions of dollars to promote digital “personalized learning,” a data-friendly 
approach to pedagogy that also demands continuous assessment.” 


The No Child Left Behind testing regime is now widely considered to have been ill-advised 
and there is little, if any, credible research that indicates digital learning programs or virtual 
education are effective. Nevertheless, the COVID-19 pandemic has supercharged efforts to 
use digital technologies to reshape school programs.’ Despite the public relations effort that 
presents digital technologies as the common-sense solution to the dilemmas posed by the 
pandemic,‘ it is important to recognize that digital technologies also pose significant threats 
to the schools and school communities that adopt them. The assessments programmed into 
digital platforms and learning programs may negatively shape student learning, subtly alter 
the curriculum, de-professionalize teachers’ role, and appropriate and misuse student data 
unless school leaders make careful decisions. 


To understand the nature of the problem, it is important to recognize that digital platforms 
and learning programs implement particular theories of learning and child development. 


http://nepc.colorado.edu/publication/virtual-learning 


The learning opportunities these platforms offer to students are, therefore, necessarily de- 
termined by these theories, as are the assessments used to evaluate students’ accomplish- 
ments. 


Digital platforms and programs provide a variety of features that streamline assessments 
and save teachers time. For example, they may offer assessments, coordinated with content 
units, that automatically evaluate and record student performance. This is a mixed bless- 
ing, because the more that platforms and learning programs automate assessment and re- 
cord-keeping, the more they limit teachers’ ability to assess students based on their direct 
observation and impede teachers’ ability to critique or correct judgments made by the soft- 
ware. At the same time, built-in assessment programs generate data that often flows back to 
parent companies that may use it for unknown purposes. To help school leaders make sound 
decisions, we have identified six key digital assessment-related issues for them to consider. 
Below we discuss the importance of each. 


Pedagogical theories embedded in digital platforms and learning pro- 
grams shape the student learning environment. 


Many digital platforms and learning programs rely on the same behaviorist theory of learn- 
ing as did the “teaching machines” promoted for school use over 70 years ago.5 In essence, 
the approach relies on the assumption that there is a uniform set of facts or skills that stu- 
dents must master, and that knowledge of these facts or skills can be broken into small ele- 
ments and presented bit by bit to students, who can then learn each element and be tested 
on it. Students’ ability to provide the required responses to assessment questions about each 
element is assumed to demonstrate their competency/mastery of the element, and there- 
fore, “learning” is a process that repeats itself continuously until a student has “mastered” 
the presented elements.° While this kind of approach allows for students to move through 
the program at their own pace, it also assumes that children do not require a meaningful 
context for their learning.” 


“Competency-based education” (CBE),® based on this hyper-rational behaviorist approach, 
is programmed into many digital learning platforms and programs, particularly so-called 
“personalized learning” programs. These programs embody tacit assumptions. The first is 
that their designers and programmers can effectively organize the fragments of information 
students are expected to master and program the assessment tools to measure whether or 
not they have been mastered. The second is that this programming “personalizes” learning 
for all the students. Given the diversity of students’ backgrounds, needs, and learning con- 
texts, these assumptions are unwarranted. While it is true that some children may quickly 
demonstrate “mastery” of the facts and skills defined in these programs by learning how to 
answer assessment questions correctly, it is also quite possible that because they have not 
learned those facts and skills in a personally meaningful context, they will not be able to ap- 
ply their “mastery” in real-world situations. As a result, what they learn may be of little or no 
use to them—except to pass tests.? For example, a high school student who appears to have 
mastered all the required math competencies may not understand how the interest owed on 
their credit card debt compounds. 
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In contrast, when teachers help students contextualize learning through classroom tasks 
and discussions of their experiences, their understanding can deepen because they engage 
with the curriculum in a personally meaningful way. To the extent that a digital platform or 
learning program minimizes teachers’ ability to contextualize learning for their students, 
assessment evidence of student learning may be illusory. Products that encourage teachers 
to contextualize students’ learning and to conduct their own assessment of students’ under- 
standing may require more teacher time and effort than products that provide content and 
assessment. They are more likely, however, to facilitate meaningful learning and assessment. 


Assessments programmed into digital platforms and learning programs 
can shape and narrow the curriculum. 


The assumptions of digital, competency-based education—that knowledge can be broken 
into logically structured elements, that student mastery of each of those elements must be 
continuously assessed, and that constant data reporting is necessary to ensure children’s 
progress—inevitably narrows the curriculum and limits teachers’ options. The assumption 
that acquiring a collection of small bits of discrete information and numerous discrete skills 
is the essence of learning necessarily also tends to exclude anything that cannot be reduced 
to a quantifiably measurable standard. The more that teaching and learning are shaped by 
the collection and use of easily quantifiable data points, the more limited the curriculum 
and definitions of “achievement” become, and the more likely that success will be defined by 
those things that can best be captured and sorted electronically.*° Necessarily, students will 
focus their efforts to strive to succeed at those things. 


In contrast, educators have for years developed approaches to curriculum to help children 
cultivate a wide variety of interests and skills difficult or impossible to quantify. To imag- 
ine alternatives and find creative solutions to problems. To interpret information based on 
sound reasoning. To develop personal identity and use knowledge in personally meaningful 
ways. And, to develop the interpersonal and social skills necessary to participate in and con- 
tribute to democratic civic life." 


It is obvious that children can learn much more in school than predefined skills. They can, for 
example, learn to be part of a classroom community in which academic knowledge, technical 
competence, social skills, and personal identity are also developed in the context of genuine 
engagement with other people." For instance, children who learn about plant growth by co- 
operatively designing and cultivating a class garden and then eating the resulting fruits and 
vegetables have a vastly different learning experience than children who acquire informa- 
tion about photosynthesis in programmed bits and pieces, even if those facts are delivered 
by an amusing, gamified educational application. 


Since human learning is often not sequential or even logical, narrowing children’s educa- 
tion to the acquisition of one skill, fragment of information, or concept after the other in an 
apparently logical progression not only constrains their experiences, it can also undermine 
their ability to integrate what they have learned in real world situations (that is, to transfer 
their learning) and inhibit their achievement of broader educational goals.'s Digital learning 
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programs define how teachers, students, and administrators interact—by defining how they 
understand what “learning” means, what “counts,” and what is important.’* They also in- 
creasingly script the teaching and learning process, crowding out the kind of unanticipated 
teaching moments sparked by a student question or comment on which teachers capitalize 
even if it means detouring from their lesson plans. Such unplanned opportunities cannot be 
coded into any software. 


Finally, the reality of forcing all children to learn and be evaluated via technology-mediated 
relationships with their teachers contradicts the rhetoric associated with personalizing edu- 
cation and responding to children’s unique needs and interests—the rhetoric that has been 
used to promote digital platforms and programs. In other words, forcing all children to learn 
via digital means, with constant focus on assessment data and “mastery” as the definition of 
learning, can reasonably be seen as the opposite of child-centered or personalized. Digital 
programs that provide for more teacher latitude in organizing their curriculum and develop- 
ing their assessements are likely to be better than those giving teachers less latitude. 


Opaque algorithms that may be biased run the assessments pro- 
grammed into digital platforms and learning programs. 


Any test reflects the values, assumptions, social positions, interests, or biases of its cre- 
ators. In a simple example, a teacher described how seemingly innocuous language in a test 
question reflected the culture of the test creator and was incomprehensible to his students. 
The question asked students to identify which of a series of pictures was a “casserole.” The 
teacher noted that although casseroles might be common in Iowa, where that particular 
test originated, his young students in inner city Texas had never seen one and could not an- 
swer the question.'5 Concerns that the language or examples used in standardized tests may 
discriminate against minority group students have dogged standardized testing for years." 
They have led to calls for standardized tests to be replaced by locally derived assessments.” 
They have also caused parents nationwide to refuse to allow their children to take end-of- 
year summative examinations.‘® 


Assessments built into digital platforms and learning programs magnifiy these concerns. 
Much like standardized tests, the assessment algorithms built into educational software are 
presented as “neutral” and “scientific,” and to embody “truth” or fact.!° They cannot be neu- 
tral, however, because they are created by people—and people are not neutral.?° 


Algorithms are much more problematic than standardized tests because they are central to 
the day-to-day functioning of the digital educational program. They are not limited to end- 
of-year summative assessments, but rather implement the regular formative assessments 
designed to mediate between teachers and children, and to influence children’s experience 
of the curriculum. Algorithms are also less transparent than any physical assessment docu- 
ment. Unlike the example in which the teacher was able to flag the question about the cas- 
serole as inappropriate for his students, teachers may not even see the questions that their 
students are asked to answer. Yet some programs require teachers to, “in real time,” adjust 
their teaching to the results that the algorithms report. Programs that feature “adaptive” or 
“personalized” learning bypass teachers completely and automate the instructional deci- 
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sion-making that teachers would ordinarily control. 


Embedding instructional and other educational decisions in digital learning programs also 
reduces parents’ ability to advocate for their children. Unlike with summative standardized 
testing, parents cannot opt out of the assessments embedded in the digital learning program 
the school has chosen. And unlike a traditional class in which parents can question teachers’ 
decision-making if they have concerns, the more that instructional decisions are transferred 
to algorithms, the less parents are able to question. The teacher may not be able to explain 
how the algorithm works. To be clear, marketing materials for digital platforms and educa- 
tional programs portray the role of their algorithms in determining what and how a child is 
taught as an advantage—but it is not. 


Far from being “objective,” algorithms reflect the myriad choices their developers make. 
They are vulnerable to significant and difficult-to-correct error.” An algorithm that assesses 
a student’s level of understanding based on, for example, his or her pattern of responses, 
response times, and keystrokes generates conclusions based on a theoretical mathematical 
relationship between those raw data points and the student’s psychological state of under- 
standing. The key word here is theoretical. For example, essay scoring algorithms imple- 
ment a theory that high-quality essays are characterized by grammatical features such as 
sentence length, vocabulary, spelling, and subject-verb agreement. Researchers analyzed 
automatic essay scoring programs (e.g., the Educational Testing Service’s “e-rater” that is 
used to grade several statewide assessments, the Graduate Record Examination (GRE), and 
the Test of English as a Foreign Language [TOEFL]?*) by having them score nonsense essays 
composed of strings of sophisticated words and sentences that made no sense. The nonsense 
essays consistently received high, sometimes even perfect, scores.”3 


Companies’ proprietary assessment algorithms are rarely, if ever, offered to external review- 
ers to analyze.*4 Therefore, the validity of the content and of the assessments those algo- 

rithms generate cannot be challenged by the students 
Far from being “objective,” who are subjected to them. It must simply be accept- 
algorithms reflect the myriad ed as “true.”* The students’ role is simply to “master” 


choices their developers make. what is presented to them and accept the rulings gen- 
erated by the algorithms. 


Automated grading and record-keeping are promoted as ways to decrease drudgery and in- 
crease teacher time with students. However, digital platforms and learning programs actu- 
ally marginalize teachers by taking the critical matter of assessment and the content of con- 
versations about learning largely out of their hands. For example, teachers may be unable to 
see how their students earned the designation of mastery of a skill or achieved a goal in some 
applications because the software, not the teacher, has determined the questions asked and 
the grades assigned. If the software and its assessments are biased and have limited validity, 
the teacher would never know. Neither would the children, their families, school adminis- 
trators, employers, or anyone else who later gets access to the software’s output. The more 
that a digital platform or learning program inserts itself into the relationship between stu- 
dents and teachers, the more opportunities there are for its output to be biased or flawed, 
and the greater the influence of those biases and flaws is likely to be on how students are 
taught and assessed. The less that it is programmed to do, the less problematic it is. 
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Assessments in digital platforms that use predictive analytics, arti- 
ficial intelligence, and machine learning can harm students in diffi- 
cult-to-identify ways. 


As companies experiment with artificial intelligence and machine learning to provide schools 
with predictive analytics, the dangers associated with the opacity of algorithms intensify. 
For example, in 2019, Instructure CEO Dan Goldsmith was discussing a new feature of the 
company’s popular Canvas learning management system when he promised the ability to 


start making recommendations and suggestions to the student or instructor in 
how they can be more successful. Watch this video, read this passage, do prob- 
lems 17-34 in this textbook, spend an extra two hours on this or that. When we 
drive student success, we impact things like retention, we impact the productiv- 
ity of the teachers, and it’s a huge opportunity.”° 


In fact, this “opportunity” puts children, parents, and teachers in a horrible bind. They have 
no way of knowing how the platform derives its recommendations, or how to evaluate their 
accuracy or worth. Their only option is to comply. 


Georgia State University uses “big data” predictive analytics to identify students who may be 
at risk for dropping out. The Hechinger Report profiled a student who the software flagged 
as unlikely to achieve the 3.5 average he would need to apply to his chosen major, nursing, 
at the end of his sophomore year.”’ Although his average was close to the cut in his freshman 
year, his similarity to other students who had not made the cut led the algorithm to mark 
him as at risk of dropping out. As a result of counseling based on the algorithm’s conclusion, 
he chose a related but less demanding major. At the time of the writing of the Hechinger 
Report’s article, he was on track to complete his degree in respiratory therapy. He did not 
drop out, but he also was pushed to abandon his original life and career goals. 


The programming of predictive analytics may very well contain “equity blind spots.”?* As 
the Hechinger Report notes, these blind spots may reinforce historical inequities and direct 
low-income students or students of color into easier majors. It is also hard to know how stu- 
dents will respond to the predictions offered by the algorithms. How many students, rather 
than lowering their goals, completing their degrees, and leading happy lives (albeit with 
lower levels of accomplishment and income than they would have had if they had achieved 
their original goal), become discouraged by the dashing of their hopes and drop out? 


In the Georgia State example, the university student made the final choice of his major. In 
K-12, the algorithm decides for students. For example, critics have questioned the validity 
of the predictions that replaced actual test scores on Britain’s spring 2020 A-level exams, 
arguing that they discriminated by race and class and caused universities to withdraw offers 
of admission.?° 


Leaders of education technology companies are bullish about their growing ability to offer 
predictive analytics and to influence student behavior and outcomes. Instructure’s former 
CEO, cited above, responded to concerns about his company’s algorithms by asking, “Should 
we take those fears of what could go wrong and completely cast aside the potential to im- 
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prove the teaching and learning experience?” he asked. “Or should we experiment and move 
forward?”’° Given the far-reaching implications of predictive analytics on students’ life out- 
comes, school leaders should avoid programs that use them. 


The economics of proprietary digital platforms and learning programs 
incentivize opacity and discourage adequate testing of their algorithms. 


Raising questions about how a given piece of software actually works is a potential threat to 
its profitability.2* An external audit of programming could, for example, flag serious prob- 
lems that throw into question the ability of the software to do what its creators claim it can 
do. This could significantly delay, if not prevent altogether, schools from adopting it. 


The proprietary nature of algorithms allows companies to conceal their programming. It 
also allows them to make stronger statements about the validity of the results they report 
than are necessarily warranted. Sara Marie Baker, former research director for a private 
healthcare consultancy, explained how this works: “The level of confidence with which you 
[as a business] can make statements or draw conclusions is greater because the data is pro- 
prietary and no one will see it. Your standards of scientific rigor are less. Even though the 
trendy term is ‘predictive analytics,’ it’s not so much causality as a reliable correlation.* ” 
This is an important warning for school leaders to consider when reviewing claims made 
about educational software. Digital platforms and learning programs that have undergone 
third-party algorithmic auditing—especially because of the economic incentives to avoid 
such review—are less likely to contain flaws that would negatively impact students. 


Digital platforms and learning programs may not adequately protect the 
student assessment data they gather and store. 


In addition to whatever educational purpose they may serve, digital instruction and asses- 
ment by their nature function as mechanisms of behavioral record-keeping. Since assess- 
ment and other school-related data are extremely valuable, there are incentives to try to 
exploit any data not properly safeguarded. In one example with far-reaching implications, 
the state of New Mexico sued Google in 2020, accusing it of using personally identifiable 
student information it obtains from school-assigned Chromebooks and G-Suite for Edu- 
cation accounts to inform its advertising business.?3 In another example, an October 2019 
breach of the Naviance college planning platform led to theft of 6,000 Montgomery County 
students’ personal information, including their GPAs and SAT scores.*4 

Given the inadequacy of current safeguards, it is not surprising that threats to student data 
continue to increase,°5 despite voluntary guidelines supported by the tech industry,°° legis- 
lation in some states,” and federal legislation in the form of the Children’s Online Privacy 
Protection Rule (COPPA),?° Protection of Pupil Rights Amendment (PPRA)?? and the Family 
Educational Rights and Privacy Act of 1974 (FERPA).*° FERPA, in particular, was weakened 
in 2008 and 2011 to allow schools to name technology companies as “school officials” and 
thereby to provide data to them without parental consent.** Contracts with companies serv- 
ing as school officials may allow for them to share data with third parties, to send students to 
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third parties without adequate data provisions, or to use student data for purposes outside 
their specified education purpose.*? Rather than simply accept reassurances that a company 
collecting student data complies with technology industry self-regulation or relevant legis- 
lation, school and district leaders would be wise to carefully examine the contracts, terms of 
service, and privacy policies to which they are asked to agree in the name of their students. 
They should also ask specific questions about what the companies they contract with do with 
their students’ data and how they protect it. 
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Research Landscape Related to Digital Platforms and/or Learning 
Programs in a Virtual Environment: Assessment 


Michael K. Barbour, Touro University California 


In a review of the research on the nature and quality of the curriculum and student experience of 
virtual education, Barbour found “almost a complete absence” of research.*? This is particularly 
true about the nature of assessment in virtual education. We therefore have to rely on limited and 
dated evidence, and on anecdotal reports. The existing evidence paints a picture of digital assess- 
ments that is less personalized than appears in marketing materials for virtual education.“4 


For example, in 2006, Klein found that the mastery-based curriculum used by the California Vir- 
tual Academy required students to achieve 80% on lesson assessments.‘ If they did not pass, they 
were returned to the lesson in order to retake the exam. She also found, at the time, that the stu- 
dent’s “learning coach” (i.e., the parent/guardian) was responsible for determining if the student 
had successfully completed the outcomes of a specific lesson. Fourteen years later, features like 
these are built in to digital platforms’ algorithms.*® Ohanian found the same type of assessment in 
her 2004 evaluation of the K12 history curriculum for kindergarten through second grade: 


Furthermore, the claim that lessons are adapted to the needs of each student is not borne out by 
the facts. If a student misses more than 20 percent of a lesson assessment, the parent is told the 
student must repeat the lesson. If the student again misses more than 20 percent, the instruction 
is to repeat the lesson again. And again. The so-called “needs of each student” is an endless loop of 
repetition of the same material.*” 


The same approach to assessment of content knowledge is described as part of the Summit Learn- 
ing Program, a nationally marketed “personalized learning” program. Students take 10-item, com- 
puter-generated assessments on each section of content, which they repeat until they answer eight 
items correctly.4® 


The problematic nature of assessment in virtual education has been raised in the literature with 
both full-time and supplemental settings. For example, in a 2016 study of an online credit recovery 
program in North Carolina, Stallings and his colleagues found little difference in short-term suc- 
cess rates (as represented by, for example, end-of-course exam scores) between the online credit 
recovery students and other credit recovery students in the state.49 When they examined graduation 
rates as a measure of longer-term success, they found that online credit recovery students were 
less likely to graduate than other credit recovery students. Those online students who did graduate 
were more likely to graduate within four years, however.®° Further, Heppen and her colleagues’ 
2016 study of Algebra 1 credit recovery in Chicago Public Schools found students in an online credit 
recovery to report lower confidence in their mathematical skills than students in face-to-face credit 
recovery classes.5* 


The deficiency on long-term measures of success and the lack of confidence among online students 
suggests that the process of presenting small elements to students bit by bit may have helped some 
of them pass a mastery-based assessment immediately following the virtual instruction, but had 
little lasting impact on their knowledge or understanding of the overall curriculum. 
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Conclusion 


The adoption of commercial digital platforms and learning programs poses real risks to the 
integrity of student assessment. School and district leaders can minimize the risks by judi- 
ciously choosing and using products they bring into their schools. To minimize the risks, it is 
important that school educational programs properly frame consideration of any technology 
considered for adoption by ensuring that digital platforms and learning programs do not 
drive the curriculum, pedagogy, assessment, or data collection and record keeping practices 
of the schools. In order to properly determine whether and in what manner to adopt a digital 
platform or learning program, we recommend that school and district leaders consider: 


The pedagogical values, goals, and practices they hope to achieve before considering 
the adoption of a particular digital educational product; 


The ways in which any digital educational product would advance their self-defined 
values, goals, and practices; 


The potential negative consequences—in this case, for assessment—that may be asso- 
ciated with the use of that product and devise strategies for avoiding them; 


Which of their defined values, goals, and practices can be best achieved by non-digital 
means and which require digital means; 


As they assess the suitability of any particular project, we recommend that they consider: 


The pedagogical theories built into the product’s assessments; 
The ways that the product’s assessments may shape and narrow the curriculum; 


Cultural and other biases that may be embedded in the algorithms that run the prod- 
uct’s assessments; 


The dangers associated with predictive analytics, artificial intelligence, and machine 
learning; 


How the economics of digital platforms and learning programs may increase their 
opacity and discourage appropriate pre-implementation testing of them; and 


How the product gathers, stores, and protects student data created as a function of its 
assessments. 
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When schools import proprietary digital technologies, there is a risk that the companies 
involved may exploit student data. Any app or website can easily incorporate technology to 
collect IP addresses and other information, including which pages, content or ads children 
see or click on; what they download; what games they play; what device a child is using, with 
what operating system and settings, and so on. Educational technology platforms, partic- 
ularly those from companies with contracts defining them as “school officials,” can access 
even more data, including data from school data systems.' Given the economic value of data 
in the surveillance economy, any bit of information that can be collected is collected.? Such 
comprehensive information facilitates behavioral tracking, which can be used in current and 
future product-related research, as well for other unspecified purposes.? 


Schools and districts now routinely collect, store, and report data for state longitudinal data 
systems on such things as attendance, tardiness, test scores and grades. Teachers record 
student behavior in classroom management applications and use “personalized” or “adap- 
tive” learning technologies that record student keystrokes, answers, and response times as 
they work their way through the curriculum or take assessments.* The U.S. Department of 
Education actively encourages the use of massive student data sets (commonly referred to as 
“big data”5) to facilitate technological “innovation” on the largely unsubstantiated premise 
that it will lead to “deeper learning” and better assessment and support systems.° 


While such massive amounts of specific and personal data are being collected about children 
at school, it is rarely clear how all this information may be used in the future. It may be used 
to support student learning or direct students to resources. It may also be used to manipu- 
late students, cultivate them as current and future consumers, or sort and evaluate them for 
purposes unknown and unapproved by their schools or parents.” Corporations that gather 
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information from children in an educational context may claim not to use it for commercial 
gain, but there are no guarantees.® A 2018 Fordham Law School study of data brokers’ sale 
of student lists found a wide variety of student information for sale—including a list of 14- 
and 15-year-old girls for family planning purposes.° The researchers were largely unable to 
discover the sources of the data for sale.’° Moreover, the Federal Trade Commission noted 
that the resale of data is so common that it may be virtually impossible for consumers to 
determine the origin of any commercially available information about them.” 


School contracts with digital vendors often include provisions that prohibit selling or trans- 
ferring data, or using the information for purposes other than its stated educational use. 
However, those provisions can often be insufficient to actually protect the data from misuse 
by the companies that collect it or by their partners.'? And, since data are fungible, it would 
be surprising if some companies do not collect and conserve data in order to, for example, 
increase the company’s value to a prospective buyer. 


Data security is also a concern.’ High-profile breaches and hacks demonstrate that many 
education technology applications lack adequate data security to protect the student data 
collected.'* In a 2018 report, the Federal Bureau of Investigation (FBI) noted that the “wide- 
spread collection of sensitive information by EdTech could present unique exploitation op- 
portunities for criminals,” and that education technology connected to the internet could 
facilitate criminals’ access to data children’s devices collect for education purposes.’ 


Given the massive amounts of student data collected and the threats to student privacy that 
virtual technologies pose, it is essential for school leaders to carefully review the privacy im- 
plications and data safeguards of any digital platform or learning program being considered. 
Six key issues related to student privacy are discussed below. 


Digital platforms and learning programs may share student data with 
third parties for unknown purposes, or in other ways fail to adequately 
protect student data. 


It is this simple: Data that are not collected and/or stored are not available for misuse or 
theft. For this reason, a product that collects minimal data is preferable to a product that 
collects more. Consider carefully whether the analytics a product offers are really necessary. 
Avoid the temptation to purchase a product that offers analyses that you do not want to use 
now, but “might” want in the future. To protect student privacy, it would be preferable to 
choose a product that avoids collecting any data that you do not have a specific, immedi- 
ate, interest in having. As a side benefit, a “no-frills” product may be less expensive than a 
product containing bells and whistles you probably won’t use and that puts your students at 
greater risk. 


Private vendors may have the slickest marketing materials, but they are not necessarily the 
best choice to provide virtual learning strategies. In their 2019 study of virtual schools, 
Miron and Elgeberi found that districts have been increasingly creating their own virtual and 
blended schools, and that those schools’ students perform better on state assessments than 
students attending charter virtual schools—especially compared to charter schools managed 
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by for-profit education management organizations.’ Also, unlike private companies, school 
districts have no financial incentive to collect and store excess student data. 


How proprietary digital platforms and learning programs operate is 
rarely, if ever, transparent. 


Algorithms are procedures for solving a mathematical problem in a finite number of steps.’” 
In software applications they are the formulas that collect, sort, and organize data. The 
programming of privately developed algorithms is largely hidden from the public behind 
the legal veil of “proprietary information.” As a consequence, there is no way for either the 
individuals or the institutions to know what data are being collected or what is done with 
those data, except as the provider may choose to share that information.'® Until regulators 
require that the programming in software products used by schools be transparent and re- 
viewable, the ability of school leaders to learn how a product works is limited to their power 
to walk away from a deal unless they get the information and protections they demand. It is 
nonetheless important that school leaders make transparency an issue in any negotiation of 
the purchase of a digital platform or learning program. 


The contract language associated with digital platforms and learning 
program requires expert review. 


In 2017, the Electronic Frontier Foundation researched the privacy policies of 152 educa- 
tion technology services used in schools. They found that only 118 of the 152 had published 
privacy policies. Especially important with respect to data security, of that 118, only 78 men- 
tioned data retention policies and only 46 reported using encryption (and in the latter case, 
encryption tended to be mentioned with respect to billing information and not necessarily 
with respect to other stored student data). 


When privacy policies and terms of service do exist, they may contain clauses that sound 
reasonable on the surface but actually present a risk to students. The Terms of Service for 
the Summit Learning Program, for example, warn schools that their use of the services is 
entirely at their own risk, that there are no warranties whatsoever, and that they waive any 
right to a class action suit and agree in advance to binding arbitration.”° 


Our examination of platform privacy policies found vague disclosures of how the vast 
amounts of information collected from children and teachers would be used. The company 
Instructure, for example, uses the information collected from its virtual learning platform, 
Canvas, to improve websites, apps, and services, and to “personalize and improve” users’ 
experience with the platform. Companies may also share aggregated and so-called “de-iden- 
tified” data without notice to users, despite evidence that such de-identified data is easily 
re-identified.?! Pearson’s Schoolnet is designed to collect and hold data on every assessment 
children take in their classes and for district and state testing purposes, with no published 
privacy policy for parents to evaluate.?? How data collected by these digital platforms may be 
used in the future is unknown. 
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We do have some hint, however, of the extent of the possibilities for exploiting student 
data. Companies using predictive analytics are already collecting and combining data from 
assorted sources (including insurance claims, digital health records, housing records, and 
personal information about a person’s friends, family and roommates) for use in algorithms 
that produce “risk scores” to identify individuals at risk of opioid addiction or overdose. 
These scores are sold to doctors, insurers and hospitals to be used in their decision-mak- 
ing.?3 Further, several hundred education technology companies partner with Amazon Web 
Services (AWS) in an initiative called EdStart. Marachi and Quill noted that although these 
companies may promise compliance with U.S. data privacy laws, once data are collected and 
combined across international borders, companies may no longer be held to the laws of the 
country where the data were originally gathered. Stored in international servers, the data 
may be transferred or sold without any oversight.*4 


Digital platforms and learning programs often send students to 
third-party sites whose content and privacy policies have not been ade- 
quately vetted. 


When children enter the Internet environment, even if they enter from a responsible site 
with a thorough and transparent privacy policy, they are quickly exposed to other commer- 
cial sites that may be less concerned about their privacy. As they move around the Internet, 
using educational sites and jumping off from them to surf or play on other sites, their ac- 
tivity is constantly tracked and recorded for future use.*5 Because these data are not part of 
the “educational record” protected under the Family Educational Rights and Privacy Act of 
1974 (FERPA), they may be used to target marketing to children and their families, or to 
build profiles that would be of interest to such potential purchasers as colleges, universities 
and businesses that seek to market products to students, as well as to potential employers 
or military recruiters.”° 


Digital educational technology provides the opportunity for students to take breaks by shift- 
ing to additional sites. But some products actively direct students to other sites.2”7 Summit 
Learning and Canvas, for example, connect children to third-party sites (such as YouTube) 
that collect data for advertising purposes. Both Summit Learning and Instructure (Canvas’s 
parent company) deny responsibility for any use a third party might make of children’s or 
teachers’ data. YouTube is not part of the educational suite of applications that Google offers 
to schools. The implication is that YouTube tracks users, regardless of whether they arrived 
at YouTube from an educational site or even from one of Google’s educational applications. 
Parents are thus in the impossible position of being responsible for reviewing the lengthy 
and often incomprehensible privacy policies of the numerous third-party sites or agreeing to 
their terms with no understanding of the implications. They are then further responsible for 
independently negotiating with their schools and districts if they are unwilling to have their 
children be subject to policy provisions. This is literally impossible for virtually all parents. 


Thus, when a cloud-based learning management system, such as Canvas, sends students to 
multiple third-party sites, multiple vendors gain access to student browsing information 
(e.g., what the students view and metadata about their interactions). This creates such a 
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complex set of dynamic relational data drawn from multiple sources that it is impossible for 
students or their families to verify or even be aware of data being circulated about them.*® 


“De-identified” student data can be easily re-identified. 


As noted above, the digital technology industry promotes data de-identification (also called 
anonymization) as the solution to concerns about tracking.?? Even if student data is de-iden- 
tified, however, students’ personally identifiable information (PII) may not be fully or per- 
manently protected. 


Using only de-identified behavioral tracking data, marketers can target a given computer’s 
user with advertisements and other communications geared specifically to appeal to and 
influence that user. Google, for instance, has repeatedly been accused of doing exactly this. 
The state of New Mexico sued Google in February 2020 for violating the Children’s On- 
line Privacy Protection Act (COPPA). 2° The suit accuses Google of using school-assigned 
Chromebooks and “G Suite for Education” accounts to illegally collect information including 
students’ online behavior, location, voice recordings, contact lists, and passwords. It further 
accuses Google of using the personal information it illegally collects for advertising purpos- 
es. When the child is the primary or only user of the device (as is certainly the case when 
that device is a school-assigned Chromebook, for instance),?' marketers do not need student 
identification at all in order to target specific students. 


This being the case, the editor of the trade publication Advertising Age, Ken Wheaton, blunt- 
ly called data de-identification “a load of horseshit . .. a clever bit of technical and verbal 
misdirection used by marketers and tech people to keep regulators at bay.” 3? He explains, 
“You might not know my name (but you probably do), but that hardly matters if you know 
every move I make, every breath I take.”33 


Computer scientists and data experts have known for over a decade that complex de-iden- 
tified datasets—such as student datasets—can easily be re-identified.*4 If a handful of dat- 
apoints in an de-identified dataset match a handful of datapoints in another, identified 
dataset, the de-identified data are no longer anonymous. For these reasons, school leaders 
should not be reassured by promises that student data is de-identified. Instead, they should 
ask questions about the nature and amount of de-identified data held by the vendor of any 
product they are considering, what those data are used for, how they are protected from 
misuse and theft, and how and when they will be destroyed. 


Digital platforms and learning programs may not adequately secure stu- 
dent data. 


It is more effective, but more expensive and therefore less common, to incorporate security 
into technology development from the beginning of a project rather than at its end.*5 It is 
also expensive, and therefore less common, to correct issues that may be unearthed by an 
algorithmic audit. For these reasons, the number of security breaches in public schools is 
growing. The Cybersecurity Research Center counted 348 cybersecurity incidents in 2019 
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alone, nearly three times as many as were reported in 2018.3° Approximately half of these 
incidents resulted from the actions of insiders to the school community, primarily education 
technology vendors.?” 


Current legal protections for student privacy are extremely limited.3° Federal law theoreti- 
cally prohibits the use of data held by private companies for purposes unspecified in their 
contracts,?° and over 425 companies have signed onto a self-regulatory pledge that bans “be- 
havioral targeting of advertisements.”4° Companies are, however, unlikely to be held to ac- 
count for security breaches or for misuse of children’s data. The Family Educational Rights 
and Privacy Act (FERPA) threatens to withhold funding to schools as a result of data misuse, 
but this punishment has never actually been imposed.*! A November 2018 audit found not 
only a two-year backlog in the Department of Education’s Privacy Office’s processing of 
FERPA complaints, but also that the Privacy Office is unable to resolve many of the com- 
plaints because of “significant control weaknesses” and unresolved policy questions about 
FERPA.#? 


Citizens may bring complaints to the Federal Trade Commission (FTC) if they believe a 
signatory company has violated the Student Privacy Pledge. Like the U.S. Department of 
Education, however, the FTC seems disinclined to act decisively to censure technology com- 
panies. For example, it still has not acted on the 2015 complaint brought against Google 
by the Electronic Frontier Foundation.*3 It did, however, rule against Google in September 
2019 for collecting personal information from children on YouTube in violation of the Chil- 
dren’s Online Privacy Protection Act (COPPA). In that case, the amount of the fine levied 
was the equivalent of less than three months of the advertising revenue Google makes from 
children’s videos, prompting critics to note that in effect, Google would not be discouraged 
from violating COPPA in the future.4 


In many cases, state legislation designed to protect student privacy by prohibiting commer- 
cial use of student data explicitly exempts data collected from students for “adaptive” or 
“personalized” student learning purposes.*® Such language nullifies other clauses of these 
bills designed to prevent tracking of students, because tracking is an essential aspect of “per- 
sonalized” student learning. In other words, school and dis- 
trict leaders should hold any product they adopt to a higher 
standard than compliance with relevant state or federal pri- 
vacy laws requires. 


Current legal protections 
for student privacy are 
extremely limited. 


Because of the ease with which de-identified data may be re-identified, data experts refer to 
“Five Safes” by which data can and should be secured: Data should de-identified. Data col- 
lected should be analyzed only by trained and accredited specialists. Data analyses should 
be done in a secure setting. Data should be secured in a way that prevents unauthorized 
removal of any data. Data analyses done should be checked and confirmed as non-disclo- 
sive.4° This framework used in government and research settings is designed to provide 
comprehensive and long-term integrity of any data collected. There is no legal requirement 
for private companies to use this framework, but to the extent that they do, students’ data 
will be more effectively protected. 
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Research Landscape Related to Digital Platforms and/or Learning 
Programs in a Virtual Environment: Data Privacy 


Michael K. Barbour, Touro University California 


There has been no research in the field of K-12 distance, online, and blended learning focused on student data 
privacy practices beyond that of Boninger and her colleagues in 2019 and 2020.* The only other information 
available is from educational bloggers, investigative reporters, and whistleblowing teachers. 


For example, in 2008, Arizona-based blogger David Safier revealed that the Arizona Virtual Academy (a K12, 
Inc.-managed virtual school) had outsourced the grading of middle school, and a year later high school, stu- 
dent papers to a private company based in India in an effort to cut costs.4* According to Safier’s reports, the 
practice was revealed when parents began to question the nature of comments on the students’ work, and then 
began to complain to the virtual charter school (which appears to be when Safier first began investigating the 
issue). Safier questioned whether commenting on and/or scoring student work constituted direct or indirect 
teaching duties (Arizona law required that those with teaching duties had to obtain a Fingerprint Clearance 
Card or Fingerprint Criminal History Check). 


In a follow-up to his original blog entry, Safier reported that in addition to the Arizona Virtual Academy, nine 
additional virtual charter schools operated by K12, Inc. (in California, Colorado, Idaho, Illinois, Minnesota, 
Ohio, Pennsylvania, and Washington) also outsourced grading.*? As a part of his second entry, he outlined 
exactly how the process worked, based on the evidence he was able to piece together. Several months later the 
story was picked up by Education Week,* but the story was not distributed by the media specifically in any of 
the affected states. 


Five years later Travis Manning, a teacher activist, wrote a letter to the editor of the Idaho Press noting that 
the Idaho Virtual Academy was one of the nine other online charter schools operated by K12, Inc. that Safier 
had referred to in his original piece.*' At the time of Manning’s letter, the legislature in Idaho was debating 
K-12 virtual learning policy. In the months that followed, officials from both the Idaho Virtual Academy and 
K12, Inc. confirmed the story.” 


Interestingly, the officials claimed that it had been a small pilot project that ended rather quickly. However, 
Safier’s original investigation of the outsourcing detailed that the grading practice existed for at least 10 dif- 
ferent K12, Inc.-operated virtual charter schools for at least two school years, and it also included tutoring 
services in four states (California, Colorado, Idaho, and Pennsylvania) for an unknown amount of time. 


More recently, a group of teachers attempting to organize on behalf of the California Teachers Association 
lodged a number of complaints against the California Virtual Academies (virtual charter schools operated by 
K12, Inc.),53 including that the cyber charter school “permitted overly wide staff access to sensitive student 
data, such as psychological reports and special education status.”5+ However, the California Department of 
Education did not conduct an investigation and closed the matter due to a lack of data on the part of the com- 
plainants.® In 2016, the California Virtual Academy (2016) reported to their employees that they had “learned 
of an incident that might affect the security of your personal information,”® although there was no mention or 
additional coverage indicating that this breach may have impacted student data. 


These kinds of reports are similar to those described in the Network for Public Education’s 2018 guide, Online 
Learning: What Every Parent Should Know, as a part of a section entitled “Is Privacy Sufficiently Protected 
When Students Learn Online?”’” The section details how in Pennsylvania, K12 Inc. was violating “federal pri- 
vacy law by requiring parents who enroll their children to waive their rights to have their children’s personal 
information protected from unrestricted disclosure and/or commercial use,”5° as well as data breaches by 
companies like Schoolzilla and a growing number of schools and districts. However, beyond these isolated re- 
ports, there is no empirical research into privacy within the virtual education literature. Given this situation, it 
is incumbent on school leaders to thoroughly assess the potential risks to student privacy posed by any digital 
platform and/or learning program they are considering for adoption. 
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Conclusion 


Unfortunately, there are real risks to students’ privacy posed by any collection of data about 
them. School and district leaders can minimize the risks by making judicious choices of 
platforms and programs. To avoid introducing significant privacy threats, we recommend 
that a school’s educational program be the framework used to consider of any technology 
for adoption. In other words, technology and applications should not drive the curriculum, 
pedagogy, assessment, or data collection and record-keeping practices of the schools. We 
recommend that school and district leaders consider: 


¢ The pedagogical values, goals, and practices they hope to achieve before considering 
the adoption of a particular digital educational product; 


The ways in which any digital educational product would advance their self-defined 
values, goals, and practices; 


The potential negative consequences—in this case, for student privacy—that may be 
associated with the use of that product and devise strategies for avoiding them; 


Which of their defined values, goals, and practices can be best achieved by non-digital 
means and which require digital means; 


As they assess the suitability of any particular project, we recommend that they consider: 


¢ The extent to which, and for what purposes, the product collects, stores, and shares 
student data; 


The transparency of the product’s operation; 

The details of privacy-related contract language associated with the product; 
Whether and to which third-party sites the product directs students; 

What the product does with de-identified data; and 


How the collected data are secured. 
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